In many industrial fields, in particular in the field of packaging, a problem exists where products that are unevenly distributed on a running surface, e.g. a conveyor belt have to be detected with respect to their positions and optionally also with respect to their height, and additionally in top view of non-circular products also with respect to their rotational position and in top view of non-rotational symmetrical products, their geometrical shapes or surface features. Thus, robots arranged along the conveyor belts, so-called pickers can grip the products in a controlled manner, lift them up, and e.g. place them in a box in a controlled manner.
So far this was performed through an incident light line scanner that runs in lines transversal over the width of the surface wherein the line scanner takes line-shaped pictures in a tele-centric manner, thus, orthogonally directed onto the surface, wherein the pictures are sequentially arranged by the processing unit and generate a continuous image of the surface in view of the running speed of the surface in that the products have a different color than the surface itself so that the products and, thus, their positions can be detected. Thus, due to conveyor belt contamination a contrast between product and surface may not suffice any more for precisely locating the product.
This method, however, was not suitable as a matter of principle to determine an elevation, thus, a thickness of the object. When the surface of the product was problematic from an optical point of view, thus, glossy, reflective, little color-contrast relative to the base surface etc., this optical detection sometimes was not sufficient for position determination. In particular it did not suffice when the reflecting surface had oriented reflective characteristics, thus, like an aluminum foil or a metal surface.
This applies in particular for situations where flat products like for example cookies or elongated rod-shaped products can partially also rest on the surface on top of each other.
Thus, the contour may facilitate detecting that there is not only one object but two partially overlapping objects, but it is not always detectable which of the two respective objects is the lower object and which is the object that sits on top, and how tall it protrudes.
This information, however, is relevant for the subsequent pickers, so that a stack or pile of products can be disassembled by several pickers arranged in sequence starting with the uppermost product.
There certainly already are methods for detecting the 3-dimensional shape of products also on a running belt.
During stereoscopy two cameras take images of a scene from different viewing angles. It is then attempted to find textured brightness distributions in both pictures, wherein the brightness distributions are coincident and define an associated portion. Making these portions overlap (cross-correlation) facilitates determining a height of these portions from the linear movement that is required to produce coincidence, but this works only where a texture is provided. For all other areas, the height remains undefined.
During strip projection a defined strip pattern is radiated on and pictures are taken from another viewing angle. Triangulation facilitates detecting a position of each point in space in the strip pattern.
However, the pattern is ambivalent due to its repeat structure. However, in order to be able to simultaneously reach a large elevation range and a tight support point density (resolution) typically several patterns have to be projected onto the scene and scanned in sequence.
Therefore, the method is not suitable for moving scenes.
Furthermore, this method requires an environment that is as dark as possible and still apparent edges in the strips cannot always be differentiated from real object edges.
During shape from shading objects are radiated with different illuminations from different directions and the elevations of a scene can be determined from the resulting shading through complex algorithmic methods.
Strongly structured surfaces and finely structured surfaces are hardly detectable through this method, and surfaces with inferior optical properties, e.g. reflection are also hardly detectable.
Even when this method is applicable, it is hardly being used on an industrial scale due to its high level of complexity, but it is rather being used in the research, like e.g. in astronomy.